Coding for Computation: Efficient Compression of Neural Networks for Reconfigurable Hardware

As state of the art neural networks (NNs) continue to grow in size, their resource-efficient implementation becomes ever more important. In this paper, we introduce a compression scheme that reduces the number of computations required for NN inference on reconfigurable hardware such as FPGAs. This is achieved by combining pruning via regularized training, weight sharing and linear computation coding (LCC). Contrary to common NN compression techniques, where the objective is to reduce the memory used for storing the weights of the NNs, our approach is optimized to reduce the number of additions required for inference in a hardware-friendly manner. The proposed scheme achieves competitive performance for simple multilayer perceptrons, as well as for large scale deep NNs such as ResNet-34.

Graph-based Algorithms for Linear Computation Coding

We revisit existing linear computation coding (LCC) algorithms, and introduce a new framework that measures the computational cost of computing multidimensional linear functions, not only in terms of the number of additions, but also with respect to their suitability for parallel processing. Utilizing directed acyclic graphs, which correspond to signal flow graphs in hardware, we propose a novel LCC algorithm that controls the trade-off between the total number of operations and their parallel executability. Numerical evaluations show that the proposed algorithm, constrained to a fully parallel structure, outperforms existing schemes.

Linear Computation Coding: Exponential Search and Reduced-State Algorithms

Linear computation coding is concerned with the compression of multidimensional linear functions, i.e. with reducing the computational effort of multiplying an arbitrary vector to an arbitrary, but known, constant matrix. This paper advances over the state-of-the art, that is based on a discrete matching pursuit (DMP) algorithm, by a step-wise optimal search. Offering significant performance gains over DMP, it is however computationally infeasible for large matrices and high accuracy. Therefore, a reduced-state algorithm is introduced that offers performance superior to DMP, while still being computationally feasible even for large matrices. Depending on the matrix size, the performance gain over DMP is on the order of at least 10%.

A High-Level Comparison of Recent Technologies for Massive MIMO Architectures

Since the introduction of massive MIMO (mMIMO), the design of a transceiver with feasible complexity has been a challenging problem. Initially, it was believed that the main issue in this respect is the overall RF-cost. However, as mMIMO is becoming more and more a key technology for future wireless networks, it is realized, that the RF-cost is only one of many implementational challenges and design trade-offs. In this paper, we present, analyze and compare various novel mMIMO architectures, considering recent emerging technologies such as intelligent surface-assisted and Rotman lens based architectures. These are compared to the conventional fully digital (FD) and hybrid analog-digital beamforming (HADB) approaches. To enable a fair comparison, we account for various hardware imperfections and losses and utilize a novel, universal algorithm for signal precoding. Based on our thorough investigations, we draw a generic efficiency to quality trade-off for various mMIMO architectures. We find that in a typical cellular communication setting the reflect/transmit array based architectures sketch the best overall trade-off. Further, we show that in a qualitative ranking the power efficiency of the considered architectures is independent of the frequency range.